空间红外的小型船舶检测旨在将小型船只与轨道轨道捕获的图像分开。由于图像覆盖面积极大(例如,数千平方公里),这些图像中的候选目标比空中基于天线和陆基成像设备观察到的目标要小得多,二聚体,更可变。现有的简短成像基于距离的红外数据集和目标检测方法不能很好地用于空间监视任务。为了解决这些问题,我们开发了一个空间红外的小型船舶检测数据集(即Nudt-Sirst-Sea),该数据集具有48个空间基红外图像和17598像素级的小型船上注释。每个图像覆盖约10000平方公里的面积,带有10000x10000像素。考虑到这些充满挑战的场景,考虑到这些微小的船只的极端特征(例如,小,昏暗,可变的),我们在本文中提出了多层Transunet(MTU-NET)。具体而言,我们设计了视觉变压器(VIT)卷积神经网络(CNN)混合编码器来提取多层次特征。首先将局部特征图用几个卷积层提取,然后馈入多级特征提取模块(MVTM)以捕获长距离依赖性。我们进一步提出了一种拷贝性衡量量 - 帕斯特(CRRP)数据增强方法,以加速训练阶段,从而有效地减轻了目标和背景之间样本不平衡问题的问题。此外,我们设计了一个焦点损失,以实现目标定位和形状描述。 NUDT-SIRST-SEA数据集的实验结果表明,就检测概率,错误警报率和联合交集的交集而言,我们的MTU-NET优于传统和现有的基于深度学习的SIRST方法。
translated by 谷歌翻译
近年来,在光场(LF)图像超分辨率(SR)中,深度神经网络(DNN)的巨大进展。但是,现有的基于DNN的LF图像SR方法是在单个固定降解(例如,双学的下采样)上开发的,因此不能应用于具有不同降解的超级溶解实际LF图像。在本文中,我们提出了第一种处理具有多个降解的LF图像SR的方法。在我们的方法中,开发了一个实用的LF降解模型,以近似于真实LF图像的降解过程。然后,降解自适应网络(LF-DANET)旨在将降解之前纳入SR过程。通过对具有多种合成降解的LF图像进行训练,我们的方法可以学会适应不同的降解,同时结合了空间和角度信息。对合成降解和现实世界LFS的广泛实验证明了我们方法的有效性。与现有的最新单一和LF图像SR方法相比,我们的方法在广泛的降解范围内实现了出色的SR性能,并且可以更好地推广到真实的LF图像。代码和模型可在https://github.com/yingqianwang/lf-danet上找到。
translated by 谷歌翻译
光场(LF)摄像机记录了光线的强度和方向,并将3D场景编码为4D LF图像。最近,为各种LF图像处理任务提出了许多卷积神经网络(CNN)。但是,CNN有效地处理LF图像是一项挑战,因为空间和角度信息与不同的差异高度缠绕。在本文中,我们提出了一种通用机制,以将这些耦合信息解开以进行LF图像处理。具体而言,我们首先设计了一类特定领域的卷积,以将LFS与不同的维度解开,然后通过设计特定于任务的模块来利用这些分离的功能。我们的解开机制可以在事先之前很好地纳入LF结构,并有效处理4D LF数据。基于提出的机制,我们开发了三个网络(即distgssr,distgasr和Distgdisp),用于空间超分辨率,角度超分辨率和差异估计。实验结果表明,我们的网络在所有这三个任务上都实现了最先进的性能,这表明了我们解散机制的有效性,效率和一般性。项目页面:https://yingqianwang.github.io/distglf/。
translated by 谷歌翻译
红外小目标检测是红外系统中的重要基本任务。因此,已经提出了许多红外小目标检测方法,其中低级模型已被用作强大的工具。然而,基于低级别的方法为不同的奇异值分配相同的权重,这将导致背景估计不准确。考虑到不同的奇异值具有不同的重要性,并且应判别处理,本文提出了一种用于红外小目标检测的非凸张力低秩近似(NTLA)方法。在我们的方法中,NTLA正则化将不同的权重自适应分配给不同的奇异值以进行准确背景估计。基于所提出的NTLA,我们提出了不对称的空间 - 时间总变化(ASTTV)正则化,以实现复杂场景中的更准确的背景估计。与传统的总变化方法相比,ASTTV利用不同的平滑度强度进行空间和时间正则化。我们设计了一种有效的算法来查找我们方法的最佳解决方案。与一些最先进的方法相比,所提出的方法达到各种评估指标的改进。各种复杂场景的广泛实验结果表明,我们的方法具有强大的鲁棒性和低误报率。代码可在https://github.com/liuting20a/asttv-ntla获得。
translated by 谷歌翻译
Human pose estimation has been widely applied in various industries. While recent decades have witnessed the introduction of many advanced two-dimensional (2D) human pose estimation solutions, three-dimensional (3D) human pose estimation is still an active research field in computer vision. Generally speaking, 3D human pose estimation methods can be divided into two categories: single-stage and two-stage. In this paper, we focused on the 2D-to-3D lifting process in the two-stage methods and proposed a more advanced baseline model for 3D human pose estimation, based on the existing solutions. Our improvements include optimization of machine learning models and multiple parameters, as well as introduction of a weighted loss to the training model. Finally, we used the Human3.6M benchmark to test the final performance and it did produce satisfactory results.
translated by 谷歌翻译
Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.
translated by 谷歌翻译
Blind image quality assessment (BIQA) remains challenging due to the diversity of distortion and image content variation, which complicate the distortion patterns crossing different scales and aggravate the difficulty of the regression problem for BIQA. However, existing BIQA methods often fail to consider multi-scale distortion patterns and image content, and little research has been done on learning strategies to make the regression model produce better performance. In this paper, we propose a simple yet effective Progressive Multi-Task Image Quality Assessment (PMT-IQA) model, which contains a multi-scale feature extraction module (MS) and a progressive multi-task learning module (PMT), to help the model learn complex distortion patterns and better optimize the regression issue to align with the law of human learning process from easy to hard. To verify the effectiveness of the proposed PMT-IQA model, we conduct experiments on four widely used public datasets, and the experimental results indicate that the performance of PMT-IQA is superior to the comparison approaches, and both MS and PMT modules improve the model's performance.
translated by 谷歌翻译
The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.
translated by 谷歌翻译
Given the increasingly intricate forms of partial differential equations (PDEs) in physics and related fields, computationally solving PDEs without analytic solutions inevitably suffers from the trade-off between accuracy and efficiency. Recent advances in neural operators, a kind of mesh-independent neural-network-based PDE solvers, have suggested the dawn of overcoming this challenge. In this emerging direction, Koopman neural operator (KNO) is a representative demonstration and outperforms other state-of-the-art alternatives in terms of accuracy and efficiency. Here we present KoopmanLab, a self-contained and user-friendly PyTorch module of the Koopman neural operator family for solving partial differential equations. Beyond the original version of KNO, we develop multiple new variants of KNO based on different neural network architectures to improve the general applicability of our module. These variants are validated by mesh-independent and long-term prediction experiments implemented on representative PDEs (e.g., the Navier-Stokes equation and the Bateman-Burgers equation) and ERA5 (i.e., one of the largest high-resolution data sets of global-scale climate fields). These demonstrations suggest the potential of KoopmanLab to be considered in diverse applications of partial differential equations.
translated by 谷歌翻译
A recent study has shown a phenomenon called neural collapse in that the within-class means of features and the classifier weight vectors converge to the vertices of a simplex equiangular tight frame at the terminal phase of training for classification. In this paper, we explore the corresponding structures of the last-layer feature centers and classifiers in semantic segmentation. Based on our empirical and theoretical analysis, we point out that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes, which breaks the equiangular and maximally separated structure of neural collapse for both feature centers and classifiers. However, such a symmetric structure is beneficial to discrimination for the minor classes. To preserve these advantages, we introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure in imbalanced semantic segmentation. Experimental results show that our method can bring significant improvements on both 2D and 3D semantic segmentation benchmarks. Moreover, our method ranks 1st and sets a new record (+6.8% mIoU) on the ScanNet200 test leaderboard. Code will be available at https://github.com/dvlab-research/Imbalanced-Learning.
translated by 谷歌翻译